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ABSTRACT 



Partnering for Elementary Environmental Science, a professional development program developed 
to enhance teachers’ skills and content knowledge in science education, pairs classroom teachers 
with resource professionals for one year of collaborative instruction. This paper reports part of 
the evaluation and research related to the program. Specifically it examines changes in teachers’ 
attitudes and instructional behaviors, changes in students’ attitudes toward and participation in 
science and schooling, and changes in the learning environment and community as a result of 

teachers’ involvement in the program. 



Evaluating the Effects of Environmental Science Programs on Teachers, Students and Communities; 

Partnering for Elementary Environmental Science 



Although collaborative projects between schools, businesses, agencies and universities have been 
advocated as a vehicle for professional development and education reform for the past decade, most of 
these efforts have lacked formal evaluation designs or mechanisms. This is especially true in the field of 

environmental education. As a result, the impact of these programs on instruction and student learning can 
only be surmised. 

This symposium examines the specific problems and topics related to implementing and evaluating 
federally-funded environmental science programs into K-12 school settings. Specifically, the symposium 
will compare and contrast the programs, evaluation procedures, and impacts of the Ecological Citizenship 
Program in urban Chicago and the Partnering for Elementary Environmental Science program in rural 

Ohio. This paper discusses the content and impact of the rural-based program. Partnering for Elementary 
Environmental Science. 

Background 

Professional development for the 1990s utilizes three powerful ideas that are “altering the face of 
professional development in this decade” (Sparks, 1994, p. 2). The first is the notion of results-driven 
professional development which judges the success of the professional development program not by how 
many teachers and administrators participate or how satisfied they are with the program, but by whether 
the program alters instruction in ways that benefit students. Honig (1994) points out that a problem with 
education reform is that it is not typically organized around improving teachers’ knowledge of content or 
enhancing their ability to collaborate to improve instruction. 

The second guiding notion is systems thinking, which recognizes the complex, interdependent 
interrelationships among various parts of the educational system. This suggests that collective, not 
individual, efforts which involve a critical mass of staff committed to improving student performance and 
making necessary instructional changes are needed to initiate systemic reform in science education. 
Further, it suggests that parents and members of the community must not only support education efforts, 
but be actively involved in them. 

A third notion driving professional development programs is constructivism. Constructivist 
professional development involves multiple forms of job embedded learning. It depends on successful 
networking among education professionals that focuses attention on instruction and learning, provides 
nurturing to schools, and brings schools together to broaden perspectives and offer needed collegial 
support. In short, it suggests that teachers and educators must work collaboratively toward improved 
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instmction. 



A promising mechanism ror carrying these powerful ideas into professional development for the 
‘90s is collaborative relationships or partnerships. In the early 1980s, the federal government began to 
recognize the need to integrate school and business entities. By 1989, the Department of Education 
estimated that over 140,000 school-business partnerships existed nationwide (Rigden. 1991 ). Initially, 
most businesses avoiaed getting involved in decisions which impacted curricula and educational systems. 
The partnerships, mostly confined to urban and suburban areas, took on a variety of forms ranging from 
providing equipment or vlnancial support to the school with no direct involvement with teachers or 
students, to “popping in and doing a few ‘gee whiz’ things” (Sills, Barron & Heath. 1993). It is 
uncertain, however, that these partnerships resulted in fundamental changes in instruction or student 
learning. Miron and Wimpelberg ( 1989), for example, found that only eight of the 450 local school- 
business partnerships they investigated led to instructional change. According to Cobb and Quaglia 

(1994), many partnerships achieve worthwhile objectives, but many also fail in their attempts to improve 
student learning. 

More recently, partnership efforts have focused on what Sills et. al. (1993) call the “next generation 
of partnerships.” These reform-based partnerships intend to go beyond compiling good projects to 
impacting instruction, student learning, and teacher empowerment. Successful reform-based partnerships 
tend to: 1) be innovative and pioneering; 2) involve hands-on, classroom-focused, and age-appropriate 
activities; 3) grow out of long-term collaborative relationships; 4) assess both student learning and the 
partnership process; 5) bring partners together often - a few times a month, or weekly; and 6) involve 
multi-level support. Cobb and Quaglia (1994) identify seven choices which must be made as schools and 
businesses engage in collaborative relationships. They advocate choices that move the partnership into the 
relationship domain, characterized by a dynamic focus on relationships among people rather than on 
structure, a focus on individual rather than organizational needs, self-examination, and multiple power 
bases and benefits. That is, having a content expert in the classroom does not automatically result in 
enhanced instruction and learning. This occurs “when teachers and scientists develop good working 
relationships, when they move forward together on an experiential curriculum, and when scientists 
become a normal presence in the school” (Sills et. al., 1993, p. 69). 

Yet the ability of collaborative efforts to enhance education is yet to be demonstrated. Few 
researchers have studied the dynamics of these efforts or explored how they can be improved. Cobb and 
Quaglia (1994) point out the need for research on the micro-level interactions within partnerships in order 
to ensure successful school reform. Further, the opportunity to enter into partnerships with businesses is 
largely absent from rural schools and many urban centers, v here professional development and education 
reform are often sorely needed. 



Professional Development Program 

Partnering for Elementary Environmental Science, a collaborative program involving The Ohio 
State Umversity-Mansfield, the Ohio Department of Natural Resources, and the Science and Mathematics 
Network of Central Ohio, attempts to enhance science teaching by linking teachers with namral resource 
professionals in reform-based partnerships in predominantly rural schools. During an intensive four-day 
summer institute, teachers and resource professionals are assigned to pannership teams and provided with 
instruction regarding the fundamentals of reform-based partnering. Partners work together throughout the 
institute, which models and provides training in inquiry-based environmental science instruction and 
supponing pedagogy including process skills, management of students and materials, authentic 
assessment, and questioning strategies. Throughout the summer, partnership teams work collaboratively 
to develop a year-long action plan outlining the content and strategies for integrating activity-based 
environmental science instruction into the existing curriculum. Throughout the academic year, teachers 
and resource professionals implement the action plan and work together in the classroom, assuming a 
variety of roles and responsibilities. Partnership teams are in contact with the project directors for 
encouragement and mentoring. Site visits enable project directors to observe and evaluate the progress of 
participants toward the program goals. Panicipants in the program meet for day-long seminars in the 
Autumn and Spring to share their progress, evaluate their effectiveness as a partnership team, examine the 
impact of tbcir action plan on instmction and sUident learning, and to continue planning. 

Program E valuation 

Evaluation of the Partnering program was directed toward: a) changes in teachers’ and resource 
professionals’ perceptions about science instruction and partnering; b) the impact of the partnership 
program on the instructional process; c) changes in students’ participation in and attitudes toward science, 
school, and science-related careers; and d) changes in the learning environment and community. 

study examined the impact of participation in a partnership program among teachers 
and natural resource professionals in 1 1 predominantly rural counties in a Midwestern state. The 
elementary teachers represented kindergarten through grade six and special education classrooms, usually 
involved with inclusion. Resource professionals represented divisions of forestry, geological survey, 
natural areas and preserves, parks and recreation, real estate and land management, reclamation, recycling 
and litter prevention, soil and water, public information and education, water, and wildlife from the state 
Department of Natural Resources, as well as the state university’s cooperative extension service, county 
soil and water cor servation districts, and the U.S. Army Corps of Engineers. 

Participants included teachers and natural resource professionals working collaboratively in 
partnership teams formed over the first two years of the program. During the first year, 23 
paitnerships were formed in five counties among 44 elementary teachers, 1 curriculum specialist, and 
27 resource professionals, involving 23 schools. While the effectiveness and structure of the Institute 



were evaluated during the first year, measures to evaluate the impact of the program on teacher 
attitudes and instruction were merely piloted. The instruments and procedures were subsequently 
polished and restructured for use during the second Summer Institute. Thus the quantitative results 
reported below reflect responses of participants during the second year of the program. During the 
second year, 39 teachers, 1 principal, and 24 resource professionals formed 18 partnerships in six 
counties, involving 20 schools. Qualitative data, including interviews and teacher and team self- 
repons, reflect responses from participants in both cohons of the program. 

Er . Q<?gdure. The measures and interviews given to the teachers, resource professionals, and school 
principals across two years are detailed below. 

1 ) Curriculum Use Inventory. As part of the application process and after one year of 
participation in the program, data was gathered regarding teachers’ familiarity with and use of eleven 
nationally known environmental science curricula, such as Project WILD, Project AIMS, and Sharing 
Nature with Children. Teachers were asked to complete a chart indicating if they were familiar with 
and had used each of the curricula. 

2) Sunimgr Institute Questionnaire (Pre. Post, and Delayed forms) were administered to all 
participants. All three forms of the questionnaire contained identical scaled, closed, and open-ended 
items designed to capture participants’ perceptions toward environmental science instruction and 
partnering. The Post form contained additional questions examining the effectiveness and impact of 
the Institute. The Delayed form was given to the second cohort after individuals had participated in the 
program for one year. The sample (N = 63) consisted of those individuals who completed all three 
forms of the questionnaire. Data was analyzed using SAS (Statistical Analysis System). Analysis of 
Variance (One-Between, One-Within ANOVA) was used for scaled items. Simple descriptive statistics 
were used for closed items, and responses to open-ended items were analyzed using content analysis. 

3) A focus group technique was employed to develop a more in-depth review of the 
effectiveness and impact of the Institute. Members of the focus group consisted of six Institute 
participants: four teachers and two environmental resource professionals. Members were selected by 
the Institute instructors. Criteria for selection included: a) representatives from teacher and resource 
professional groups, and b) demonstrated articulation of thought-provoking, critical views and 
opinions in either large or small group settings. The focus group interview occurred on the final day 
of the Institute, and lasted for approximately one hour. The interview was audio-recorded subsequent 
to obtaining permission from all participants. Assurances of anonymity and confidentiality were 
provided by the evaluators. The recording was transcribed verbatim; analysis was performed via 
content and cluster analysis techniques. 

4) RsflgCtion qugstlbns or writing prompts were given to all participants four times during the 
second Institute. During the first years’ Institute and throughout the year of involvement with the 
program, all participants were provided with and instructed to keep journals about the Institute and 
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subsequent implementation and partnering activities. Journals were reviewed three times during the 
year and feedback was provided to writers. This approach was not popular with program participants 
and was largely ineffective in securing detailed information about teachers’ attitudes and instructional 
changes. As a result, open-ended reflection questions were developed and used to obtain more 
specific data during the second Institute. Participants were given 20-30 minutes to respond to each 
question, and were encouraged to write a one to two page response. One reflective question, for 
example, was used to gather baseline information of participants’ teaching or presentation style prior to 
attending the Institute. It asked participants to describe their teaching or presentation style in detail: 
Think back over the past year to the times you taught or made presentations to students. In as much 
detail as possible, describe your presentation/teaching approach or style. What methods do you most 
frequently use? How long are your science lessons/presentations ? How do students generally 
respond? If you are a teacher, how much time per week do you spend doing science? Paint us a 
picture of yourself as a leacher/presenter/’ 

5) Individual and team imgrim evaluations were completed by all participants and all • 
partnership teams in November at the mini-conference. Both evaluations consisted of open-ended 
items designed to gather specific information on the implementation of the action plan and to parallel 
the reflective questions presented during the Summer Institute. Specifically, questions aimed to 
ascertain the perceived benefits and challenges of the program for the teachers, resource persons, and 
the school and community; what excited participants the most and least about partnering; and anecdotal 
accounts of class or students’ involvement with the program. One item on the individual interim report 
paralleled the teaching style question from the Summer Institute: “List and describe some ways that 
your professional life is different because of partnering. For example, in what ways is your 
teaching/presentation style different? What changes have you made in the classroom? Do you think 
differently about planning, presenting to groups, or learning? Please be as specific as possible.’’ Data 
from interim reports was transcribed by question and content analyzed. 

6) Individual and team final ^valuation reports contain scaled, closed, and open-ended items 
designed to capture panicipants’ perceptions toward environmental science instruction and partnering 
identical to the items on the pre- and post-institute questionnaire. Some open-ended questions 
paralleled questions to the interim reports about the specific benefits and challenges of the partnership 
and action plan. Team evaluations also asked the team to discuss extending their partnering activities 
into a second year and, if they desired to continue, to begin planning for the continuation. Data was 
analyzed using SAS, using Analysis of Variance for scaled items to see longitudinal changes in 

attitudes toward environmental science instruction and pannering. Responses to open-ended questions 
were analyzed using content analysis. 

7) Telephone L D l smews wjth building principals were conducted after partnership teams had 
been active for one and two ye ars in the program. Interview questions sought to determine if the 
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principal was aware of the teachers’ involvement with the program: if the pannership was active; the 
sorts of activities the partnerships were engaged in; the perceived benefits of the partnering program 
for the teacher, students, school, and community; and challenges of which the principal was aware. 
Descriptive statistics and content analysis were used, where appropriate, after interview data was 
transcribed. 

8) Telephone interviews with team leaders were conducted after two years of potential 
involvement in the program to ascertain the level of activity of the partnership (very active and 
dynamic, active and on schedule, limited activity, or disbanded); changes in the participants of the 
partnership; perceptions of why the partnership endured or disbanded; and impacts of the partnering 
program on teaching style, curriculum, students, and the way the teachers thought about teaching and 

learning. Descriptive statistics and content analysis were used, when appropriate, after interview data 
was analyzed. 

Results and Discussion 

Results will be presented as responses the four foci driving the evaluation and research effons 
for the Partnering in Elementary Environmental Science effort to date. 
aLWerc thgre cha nges in partic i p ant s ’ perceptions about .science instruction and pnrfnRring9 

On all three forms of the Summer Institute Questionnaire, pretest, posttest, and delayed forms, 
participants were asked to: a) consider one-word descriptors for environmental science and 
partnering; and b) indicate the degree to which they agreed with the term as an appropriate descriptor. 
Responses ranged from 5 (strongly agree) to 1 (strongly disagree). Tables 1 and 2 show means and 
standard deviations for these questions. 



Tables 1 and 2 



Analysis of variance (ANOVA) was performed on responses to determine differences between 
the responses of teachers and resource professionals and changes in their responses across pretest, 
posttest, and delayed forms of the instrument. Table 3 shows an F-value for each of three sources by 
variable for descriptors of environmental science. Table 4 shows an F-value for each of three sources 
by variable for descriptors of partnering. The three sources are: I ) main effects between groups 
(teachers and resource professionals) (A); 2) main effects of pretest, posttest, and delayed scores (B); 
and 3) the interaction of group and pre-post-delayed (AB). Significant differences (p<.05) in F- 
values are noted with a single asterisk. 



Tables 3 and 4 

A number of significant differences were observed in the perceptions of participants concerning 
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environmental science and partnering. Results are reported below for; 1 ) overall differences between 
teachers and resource professionals; and 2) changes between pre/post/delayed measures. 

Qvsialiilf fgrgnbg? between tgachgrs’ and resource pr o fessionals’ perceptinn<; Although no 
statistically significant differences between teachers’ and resource professional’s perceptions of 
environmental science were shown following the posttest measure, overall significant differences were 
noted between groups when the delayed measure was included. Specifically, teachers perceived 
environmental science to be more “active" and “exciting” than did resource professionals. 

With regard to partnering, teachers and resource professionals demonstrated perceived 
differences from pretest to posttest in three areas; teachers generally perceived partnering to be more 
"meaningful,” more “helpful,” and more “exciting.” These differences were sustained throughout the 
year, as indicated by the delayed instrument. 

C hange? between pre/post/delaved rpggspres. Significant changes were noted between pretest 
and posttest scores on the questionnaire for two environmental science descriptors, participants rated 
environmental science as more “exciting” and “interesting” at the end of the Institute than at the 
beginning. In addition, changes were observed in participants’ perceptions of four other qualities 
following the Institute, as measured by pretest and delayed questionnaires. Specifically, teachers and 
resource professionals perceived environmental science as less “unfamiliar,” teachers perceived 
environmental science as more “active’ and resource professionals were shown to perceive 
environmental science to be “less intimidating” and “less difficult.” 

Examination of the data indicated that significant differences were also observed between 
pretest and posttest questionnaires for several of the partnering descriptors. Participants rated 
partnering as more “meaningful,” more "exciting,” and less “unfamiliar” (ie., more familiar) and less 
“intimidating.” However, they also found partnering to be less “meaningful,” less “helpful,” and less 
exciting ’ at the end of the Institute than they did prior to the Institute. After one year of involvement 
m the program, based on the delayed questionnaire, no significant differences were observed in the 
participants’ perception of partnering as “meaningful” or “helpful.” This suggests that pnrticipants 
perceptions of the meaningfulness and helpfulness of partnering acmally increased during their year of 
participation in the program. Participants’ perceptions of partnering as less “unfamiliar,” 
“intimidating,” and “exciting” were sustained across their year of involvement. 

Ini sragtion gf professionals and ore/nost/ d elaved variahlp<; Interaction 

effects were observed in the teachers’ and resource professionals’ responses to the partnering 
descriptor, “unfamiliar.” In response to this descriptor of partnering, teachers changed dramatically in 
their perception of partnering from a view of partnering as unfamiliar to less unfamiliar (more familiar - 
pretest M = 2.91 , delayed mean M = 4.59) and similarly to less intimidating (pretest M = 3.82, 
delayed mean M — 4.74), In contrast, resource professionals showed only modest movement toward a 
view of partnering as more familiar (pretest M = 3.58, delayed mean M = 3.95) and less intimidating 
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(pretest M = 4.21, delayed mean M = 4.10) although they began the Institute with a more “familiar” 
and less ‘‘intimidating” view of partnering. 

£ 3niCiP^n . t»s Igvej of confidence in teachin g environmental science. Participants were also 
asked to rate their level of confidence with regard to teaching environmental science. Tables 5 and 6 
show the results. With the exception of “managing materials." for each of the dimensions of teaching 
environmental science on the pre/post/delayed questionnaires, significant differences were indicated by 
the participants’ responses. Table 6 indicates that teachers expressed greater confidence in “using a 
variety of approaches, “ “teaching process skills.” “adapting what you are teaching to a specific age,” 
managing students, asking effective questions,” and “measuring student learning using methods 
other than traditional tests and worksheets.” Resource professionals expressed greater confidence in 
their “level of knowledge about environmental science.” 



Tables 5 and 6 



The data analysis presented in Table 6 indicates a significant change (p <.05) in participants’ 
level of confidence from pretest to posttest (ie., during the Institute) in each of the dimensions of 
teaching environmental science. Specifically, participants indicated that they were more confident 
teaching environmental science following the Institute than they were prior to the Institute. Moreover, 
the data suggests that these changes were sustained throughout the year of participation in the 
partnenng program. Interaction effects were also significant for a)”adapting what you are teaching to a 
specific age group,” and b) “your own level of knowledge about environmental science.” Further 
analysis revealed that the interaction effects could be attributed to the more dramatic increase in 
teachers level of confidence when compared to the increase shown by resource professionals. 
Concerning their “level of knowledge about environmental science.” although teachers began 
significantly lower than resource professionals, on the posttest questionnaire they showed no 
significant difference from the resource professionals. 

Regarding partnering, participants were asked to indicate their level of confidence by selecting 
the statement that most accurately reflected their feeling about partnering. Options for the item were: 

1) Very Confident, 2) Let’s just say, “confident” and leave it at that, 3) Somewhat confident, or 4) I’d 
prefer to wait and see. As shown in Table 7, only the teachers indicated a significant change (p<.05) 
in their level of confidence toward partnering from pre- to posttest. Though the mean for both groups 
improved from pre to post, the teachers’ level of confidence showed significant change (ie.. 
improvement). The mean response for both groups at the end of the Institute was “ves'y confident.” 



Table 7 



10 



O 1 . 

ERIC ! i 









bl.What is the impact of the partnering nrogra m on ihe instructional prnr;ftsc9 

To begin to determine changes in classroom instruction as a result of the partnership effort, the 
following open-ended responses were content analyzed: ! ) teacher descriptions of their instructional 
approach when teaching science from the Instimte reflection question; 2) teacher responses to the 
individual interim report question asking how their instmction had changed, 3) team responses to the 
intenm and final evaluation questions soliciting anecdotal stories about how the project had benefited 
students, 4) interviews with building principals after about the benefits of the program for teachers 
one and two years of participation in the program, and 5) pre-institute and end of year curriculum 
inventories. 

Approximately three-quarters of teachers shared that their instructional approach had changed, 
sometimes radically. Most of these teachers described their approach as “less traditional”, and noted 
that they were using more “hands-on approaches”, and “cooperative groups and teaming.” They 
descnbed their instruction as less teacher-centered and textbook dominated and, although they avoided 
the “inquiry” label, many described inquiry learning taking place in their classrooms as'^students 

investigate, reason through questions that evolve during the lesson, and draw conclusions. One 
teacher shared: 

“I have always been most traditional in my teaching, but after 
this summer and with the team approach we used I’m finding I 
cga use teams within my class and I can live with talking 
because excitement is being shown and learning is taking place. 

No, it s not pencil and paper -- but it’s really not fun and games - 
- it’s learning from peers in many ways even if it is noisy it is a 
learning atmosphere. The class investigates — not just reads, 
outlines, etc. There are many answers and it is fun to see these 
reasonings for conclusions.” 

Interviews with building principals reinforced teacher self-reports of changes in their instnictional 

approaches. Principals noted more use of hands-on projects and lessons and use of cooperative learning 
and teaching. 

One theme related to instructional changes that was repeated in teacher responses was changes in 
how they plan and think about the goals of the lesson and the desired learning outcomes. Teachers 
reported integrating across subject areas more frequently and with greater ease and depth, and this report 
was reiterated by principal observations. Many teachers described themselves as more organized and 
structured and more confident. Two teachers commented: 

I plan now with vision as to how I can bring other members of 
this team’s (ie, the partners') expertise to my class and don’t 
worry so much that I have to cover X number of pages.” 

Because of the hands-on activities and the resource people in 
my classroom. I approached the whole year differently. I have 
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become a lot more confident in the areas of integration of the 
curriculum, hands-on activities, cooperative learning and less 
‘paper test taking,’ lecture teaching, and holding narrow goals." 

A second change noted in teachers’ instructional approaches, though less frequently identified, 
was an awareness and use of a wider variety of resources. Not surprisingly, this included a variety of 
the curricula and science materials with which they were acquainted during the Instimte. Table 8 
summarizes the data about teachers’ familiarity with and use of leading environmental science curricula 
reported during the application process and after one years’ involvement with the program. Table 8 
shows consistent dramatic increases in the number of curriculum programs which teacher have heard 
of, showing increased awareness of potential environmental science resources. Further, in comparing 
teachers’ use of the curricula prior to the Instimte and during their first year of involvement with the 
partnering program, teachers’ reported use of the programs increases from 4 % to 51%. This suggests 
that one specific way in which teachers’ instructional approaches changed was with increased use of 
recognized environmental science curriculum materials during their first year of involvement with the 
program. 



Table 8 



Moreso, however, teachers identified a new awareness and use of human resources. Many 
cited looking beyond the resource people collaborating with them to others in the community. One 
teacher explained: 

‘‘Another way I have changed is that I am more open to people 
who want to come into the classroom. I used to throw away 
everything in my mailbox because everi'one wanted a piece of 
my teaching time. I now see that there are valuable people out 
there excited about helping teachers. ” 

Principals highlighted this benefit of the program for participating teachers. Most principal comments 
focused on the benefits to teachers of having a resource person with whom to work. Resource 
persons were valued for their content expertise, but moreso for the impact that their relationship with 
teachers had on teachers’ professional lives. Three of the principals stated that working with the 
resource person helped the teachers become more comfortable with teaching and practice ‘‘new ways 
of teaching." In short, these principals felt. that as a result of working in a partnership team, the 
teachers were strengthened. Some principalS also cited that teachers benefited by having another 
professional with whom to collaborate. One principal also noted that the affiliation with a resource 
professional helped a new teacher acclimate to the rural environment. 

Several teachers commented on a third theme related to instructional change: that they were 
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using different methods of assessing student learning and involvement in science. One teacher cited 
having students do more “reflection" following science activities so that she had a better idea of how 
they felt about the activity and what they really learned. Teachers commented about moving away 

from “pencil and paper" methods of assessment toward journals, discussions, projects, and process- 
oriented questioning. 

Notably, few teachers commented on instraction being impacted by increased science content 
knowledge, although one teacher commented that one of the ways she had changed was that she was 
more aware of environmental issues and attempted to bring them into the classroom whenever 

possible. One principal, however, did comment that the program has broadened teacher knowledge of 
the environment, as follows. 

Teachers broadened their knowledge of the environment and 
how people and the environment interact. An example was 
learning about bees. They went on a field trip and several 
students got stung by bees. They learned a lot about how to 
treat a hornet’s nest! 

c) Were thgrg change? in student narticipation in and attitudes toward science, schnol. and scipnrp- 
rg i atgd cargcrs as a result of participation in the partnering program*? 

Although student learning was not directly assessed in this study, teachers reported increases in 
students’ enthusiasm, more positive attitudes toward science and school in general, improved 
behavior, and enhanced social skills, especially leadership and collaboration, as a result of the new 
instractional approaches. Herein, teachers often shared success stories about individual students. 

“We have a student who has severe social problems with other 
smdents-rnainly within the classroom setting. However, when 
given the chance to be involved with our environmental science 
activities, he not only was not a problem, but actually could 
articulate the conclusions and observations that were part of the 
desired outcome." 

“’^e other third grade teacher has a little guy who has a terribly 
difficult time working in small groups. We warned the resource 
person ahead of time to ‘be prepared!’ Well, you guessed it-this 
little guy got caught up in the conservation project and he was 
great! The little guy worked and cooperated beautifully in his 
group! He still straggles with group work, but this was a major 
breakthrough!” 

An unexpected and noteworthy comment made by teachers from two teams showed that their 
involvement with the partnership helped break down gender stereotypes related to science-related 
careers. One teacher explained: 

“One female student stated, vhen she found out that our 
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(female) resource person was an engineer, that “being an 
engineer is a male profession.” Positive results (of the program) 
are breaking stereotypical constraints, opening up career paths 
for students.” 

Interviews with the building principals reinforced the benefits for students which the teachers cited. 
According to one principal, the partnering program and related resources “opened up a new area to the 
students. Principals most frequently cited the active involvement in learning and the “variety of 
experiences beyond the book learning method as benefits for students in classrooms participating in the 
program. Principals noted that lessons in those classrooms extended across the curriculum and enabled 
students to connect what they were learning with real world applications. One principal noted that “even 
some discipline problems have improved because kids are active and busy.” Student response to the 
resource professional in the classroom was rated by all but one principal as “overwhelmingly positive.” 
They observed that smdents were more attentive to the resource partners because “they felt that what they 
were doing was important” and that student interest was heightened from observing resource professionals 
as career role models. Principals also frequently mentioned that students benefited from cooperative group 
work especially when it involved special needs students, and from assuming leadership roles in 
partnership-related projects. Further, they observed that students had increased awareness of the 
environment and the role they play as individuals within that environment and its conservation. 

d l.. Werg thgr? chansgs m thg learning environment a nd community as a result of particination in the 
partnering program? 

Teacher and principal responses suggest that not only instruction, but many aspects of the 
classroom climate and school environment have been impacted by the partnerships. For one thing, 
many teachers shared that “the excitement traveled throughout our entire school building.” Fewer 
principals, about one half, noted the spread of the program throughout the building. Numerous reports 
were presented of teachers from other classrooms, teacher aides, custodians, and building 
administrators becoming actively involved with the painering projects. 

In addition, teachers reported parents and neighbors of the school becoming involved in the 
partnering projects and activities. A few principals cited specific examples of parents involved as 
helpers with the partnership program in a variety of capacities. Parents helped in classrooms, served as 
group leaders and coordinators during field trips, worked with teachers and parents to build land labs 
and establish prairies, and planted trees. Parents who were involved with the program even indirectly 
were reponedly “impressed with what we’ve done.” 

While principals recalled little direct feedback from parents regarding the program, many 
teachers reported positive comments they received from parents about the new approach to teaching 
science. One teacher shared: 

“Parents have commented about the interest their students 
have developed in the environment and their ability to 
play around with science to find out an answer. They are 





14 



hooked! Science is fun and nothing to be afraid of!” 



An additional change in the school learning environment associated with participation in the 
program was teacher professional development. Although the information was not directly solicited 
from teachers or principals during this stage of the evaluation, a theme of developing teacher 
professionalism emerged. Some pannership teams identified, wrote, and secured grants to obtain 
science curricula and materials, establish land labs, or expand the partnering project throughout their 
school district. Some partnership teams reported on their activities at state-level meetings of 
professional societies, such as the Science Education Council of Ohio (SECO). the state NSTA- 
affiliate, and the Environmental Education Council of Ohio (EECO). Nearly one fourth of teachers 
received additional training in pannering and teac.her leadership skills and subsequently served as 
coaches and mentors for expanded pannership networks in their counties. A few principals noted that 
panicipating teachers and resource professionals helped with state-required curriculum revisions. 
Several teachers assumed leadership in their buildings to bring inservice training and workshops, such 
as Project WILD, to their buildings. One principal described the effons of a pannering teacher who 
shared what she had learned in the program with the building staff. The principal commented that “the 
pannership enabled our teacher to become a resource person herself.” 

General Discussion and Conclusions 

The discussion and conclusions will be delimited to two categories, a) what we have learned 
about evaluation, and b) what we have learned about the program and panicipants as a result of the 
evaluation of the Pannering for Elementary Environmental Science program. 

a) Wh^t wg’v^ Igamgd <tbQ\U gyalyaiipn. Regarding specific strategies for evaluating an 
environmental science program of this scope, three primary lessons are apparent. First, the need for 
continuity in the measures or instruments is vital if enduring effects of the program are to be accurately 
assessed. While it was imponant to pilot questions and instruments during the first year of the 
program and subsequent revisions certainly strengthened the questionnaires, important data was lost 
from an entire cohort of program participants. A similar event occurred with the second cohort, when 
questions which now seem important were omitted from the delayed form of the questionnaire. As a 
result, important program effects may be overlooked or omitted. 

Second, we learned that while they are an accepted strategy for evaluating programs, focus 
groups provide information that is too general and perhaps too positive to be of real use in critically 
analyzing and refining a program. Perhaps a “group mentality” emerges in a focus group and, 
although the evaluators are declared to be neutral and responses held in confidentiality, a positive group 
think emerges. Perhaps the bonding that has gone on among the program participants and leaders 
casts focus group participants in a role of not wanting to hurt of offend their sponsors, who are now 
friends. As a result, a different strategy was used with the third cohon of panicipants in the Partnering 
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program. In order to receive specific, analytical feedback regarding the Institute program, participants 
were broken down into “buzz groups'’ of three or four people. Within that group, participants focused 
on two aspects of the program and. following discussion among themselves for 20 minutes, 
responded in writing to specific questions. The information obtained was much more helpful in 
evaluating the Institute program than was the focus group information. 

Third, we learned that journaling simply does not work. Teachers were familiar with 
journaling and many required journals of their students, but many complained throughout the first ' 
Institute and following months about the requirement to regularly write in their journals. Resource 
persons, who were unfamiliar with journaling, found the task distasteftil either ignored the requirement 
or recorded scant information too general for any possible use or analysis. The reflective prompts 
developed for the second Institute proved much more effective. The 20 minute writing response to 
specific questions yielded more detailed, useful information from both teachers and resource 
professionals. Reflective prompts were effective in gaining self-report of baseline data on qualities 
such as teaching style which would otherwise have required hours of classroom observation. The 
piloting of reflective prompts showed that it is essential that the questions be tailored to the job 
situation or context of each group of participants. That is, similar but separately worded prompts were 
needed for the teachers and resource professionals. Also, it was important to avoid education jargon, 
especially when writing prompts for the resource professionals. 

laL What we leamgd abput t hg progra m and participants. Each evaluation strategy informed us 
about the partnering program and its participants, and was helpful in further developing the program. 
From the curriculum inventory, for example, we found out during the application process that the 
teachers were much less familiar with leading science curricula than we had expected. This led to 
major revisions in the content of the Institute: the level of instmction was moved away from a strong 
inquiry-based approach to talking more about moving teaching along the instructional continuum 
toward activity-based approaches: and we modeled and identified activities from many of the la nd ing 
curricula, briefly described those programs within the context of the Institute, and had the materials 
available for participants to review. 

Interviews of the building principals and teachers/team leaders showed very different ideas 
about the impact of the program on the school and community. Teachers saw wider influence than did 
principals and were more aware of parent reactions to and perceptions of the program. Principals 
reportedly were most aware of the impact of the program on teacher development and instructional 
behavior. They were also more conscious of high-profile activities that gained publicity than of smaller 
changes more closely related to student attitudes and learning. 

Overall, the Summer Institute Questionnaires and interim and final evaluations completed by 
individuals and teams suggest that the program is effective in accomplishing its goals. Teachers and 
resource professionals respond positively to environmental science and partnering. Initial misgivings 
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(ibout partnering expressed by resource professionals following the Institute were not evident in the 
delayed questionnaire. That is. resource professionals’ comfort with partnering increased during the 
year of participation in the program. Teachers changed more than did resource professionals in nearly 
every category, but especially in Wnfidence in teaching environmental science. More important, 
teachers grew in ways that are considered “good” science: less traditional, more integrated, more 
hands-on, more collaborative, more process emphasis, more analytical and reflective. While teachers 
grew over the year in confidence and attitudes toward partnering and environmental science teaching, 
resource professionals maintained their levels on these variables throughout the year. Notably, there 

was no wash of the positive effects of the Institute, nor were negative experiences or attitudes 
acquired during the year of participation. 

Although the effect of the partnership program on students was not directly measured, students 
did seem to benefit from participation with increased enthusiasm and more positive attitudes toward 
sciencing and schooling. They reportedly responded positively to having an additional adult involved 
in their education and serving as a role model in a science-related career. 

With less confidence, we can suggest that participation in Partnering for Elementary 
Environmental Science has resulted in changes in the learning environment and community. Data 
suggests that, at least in some cases, parents are more involved in classroom activities. In many cases, 
teachers and/or principals report professional development in teachers in their buildings, not just in the 
teachers who were directly involved with the program. This is an aspect of the program which begs 
more focused research. 

Conceiving of partnerships as a vehicle for teacher professional development and enhanced 
science instruction in rural schools appears to be supported by the evaluation of the Partnering for 
Elementary Environmental Science program. An examination of the evaluation strategies and 
procedures applied during the first two years of the program have led to a refined evaluation scheme to 
utilize on subsequent years of the program. Further, it suggests that the program is effective in 
bringing about general change in science instruction and in accomplishing many of the goals of 
science-education reform movements. What is needed, however, is a more detailed, micro-level 
analysis to fully understand the impacts of the program on teachers, students, and learning 
communities. 
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Table 1 



Environmental Science; 


Means and Standard Deviations 


bv Groups for Pre/Post/Delav. 










Variable 


Group 


Pre 


Post 


Delay 


M 


SD 


M 


SD 


M 


SD 


Environmental Science 
















Active 


Teacher 


4.71 


0..46 


4.82 


0.39 


4.94 


0.24 




Resource 


4.53 


0.51 


4.75 


0.44 


4.60 


0.50 


Unfamiliar 


Teacher 


3.29 


1.29 


4.09 


1.03 


4.21 


1.05 




Resource 


3.94 


1.00 


3.95 


1.23 


4.35 


0.93 


Intimidating 


Teacher 


3.61 


1.34 


4.22 


0.87 


4.24 


1.06 




Resource 


3.89 


1.10 


4.20 


0.89 


4.60 


0.75 


Exciting 


Teacher 


4.64 


0.64 


4.78 


0.42 


4.88 


0.33 




Resource 


4.36 


0.76 


4.63 


0.76 


4.55 


0.51 


Interesting 


Teacher 


4.74 


0.45 


4.88 


0.33 








Resource 


4.52 


0.77 


4.70 


0.47 


— 


— 


Difficult 


Teacher 


3.29 


1.14 


4.10 


1.05 


4.12 


0.81 




Resource 


3.37 


1.11 


3.60 


0.99 


4.30 


0.73 



Teachers (n = 34) Resource Professionals (n = 19) 
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Table 2 



Partnering: Means and Standa rd Deviations for Pre (U\) nnd Post (U2) 



Variable 


Group 




Pre 




Post 


Delay 


M 


SD 


M 


SD 


M 


SD 


^rtnering 

Meaningful 


Teacher 


4.67 


0.64 


4.88 


0.33 


4.79 


0.73 




Resource 


4.42 


0.76 


4.63 


0.76 


4.35 


0.93 


Unfamiliar 


Teacher 


2.91 


1.42 


4.09 


1.27 


4.63 


0.91 




Resource 


3.68 


1.11 


3.80 


1.15 


4.00 


1.26 


Complicated 


Teacher 


3.79 


1.02 


3.53 


1.19 


4.03 


0.92 




Resource 


3.58 


0.84 


3.26 


1.14 


3.30 


1.38 


Helpful 


Teacher 


4.68 


0.47 


4.94 


0.24 


4.79 


0.74 




Resource 


4.58 


0.77 


4.58 


0.77 


4.40 


0.75 


Confining 


Teacher 


4.26 


0.75 


4.28 


0.99 


4.64 


0.65 




Resource 


4.11 


0.88 


4.10 


0.85 


4.05 


1.28 


Intimidating 


Teacher 


3.82 


1.19 


4.34 


1.10 


4.73 


0.45 




Resource 


4.11 


0.88 


3.90 


0.91 


4.35 


0.75 


Exciting 


Teacher 


4.73 


0.45 


4.97 


0.17 


4.82 


0.58 




Resource 


4.11 


0.88 


4.63 


0.76 


4.45 


0.76 
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Table 3 



One-Between- One Within ANOVA by Environmental Science Variable and 
Pretest/Posttest/Delav Variable 



Variable 


Source 


df 


MS 


F 


Active 


Groups (Teachers/Resource Professionals) 


1 


1.186 


* 6.64 




Pre/Post/Delay Measures 


2 


.389 


2.21 




Groups * Pre/Post/Delay 


2 


.121 


0.69 




Error (SB/A) 


87 


.176 




Unfamiliar 


Groups (Teachers/Resource Professionals) 


1 


.911 


0.60 




Pre/PostyDelay Measures 


2 


7.895 


* 7.87 




Groups * Pre/Post/Delay 


2 


1.843 


1.84 




Error (SB/A) 


87 


1.003 




Intimidating 


Groups (Teachers/Resource Professionals) 


1 


1.182 


0.80 




Pre/PostyDelay Measures 


2 


6.308 


* 7.73 




Groups * Pre/Post/Delay 


2 


.173 


0.21 




Error (SB/A) 


87 


.816 




Exciting 


Groups (Teachers/Resource Professionals) 


1 


2.829 


* 7.36 




Pre/Post/Delay Measures 


2 


.868 


* 3.21 




Groups * Pre/Post/Delay 


2 


.024 


0.09 




Error (SB/A) 


87 


.270 




Interesting ** 


Groups (Teachers/Resource Professionals) 


1 


.721 


2.19 




Pre/Post/ Measures 


2 


1.124 


* 7.75 




Groups * Pre/Post/Delay 


2 


.013 


0.09 




Error (SB/A) 


87 


.145 




Difficult 


Groups (Teachers/Resource Professionals) 


1 


.303 


0.29 




Pre/Post/Delay Measures 


2 


9.073 


* 10.38 




Groups * Pre/Post/Delay 


2 


.648 


0.74 




Error (SB/A) 


87 


.847 





Note. *g<.05. ** Pre/Post Measures only 
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Table 4 



One-Between- One Within ANOVA bv Partnering Variable and Pretest/Posttest/Delav Variable. 



Variable 


Source 




MS 


F 


Meaningful 


Groups (Teachers/Resource Professionals) 


1 


3.137 


*6.33 




Pre/Post^elay Measures 


2 


.793 


0.18 




Groups * Pre^ost/Delay 


2 


.303 


0.52 




Error (SB/A) 


87 


.458 




Unfamiliar 


Groups (Teachers/Resource Professionals) 


1 


.002 


0.00 




Pre/Post^elay Measures 


2 


11.814 


*8.66 




Groups * Pre^ost/Delay 


2 


6.578 


*4.82 




Error (SB/A) 


87 


1.364 




Complicated 


Groups (Teachers/Resource Professionals) 


1 


5.994 


*4.56 




Pre/Post/'Delay Measures 


2 


.734 


0.68 




Groups * Pre/jPost/Delay 


2 


2.128 


1.98 




Error (SB/A) 


87 


1.073 




Helpful 


Groups (Teachers/Resource Professionals) 


1 


2.266 


*5.91 




Pre/Post^elay Measures 


2 


.270 


0.69 




Groups * Pre^ost/Delay 


2 


.293 


0.75 




Error (SB/A) 


87 


.390 




Confining 


Groups (Teachers/Resource Professionals) 


1 


.2.380 


2.48 




Pre/Post/ Measures 


2 


.448 


0.67 




Groups * Pre/Post/Delay 


2 


.394 


0.58 




Error (SB/A) 


87 


.674 




Intimidating 


Groups (Teachers/Resource Professionals) 


1 


.310 


0.23 




Pre/Post^elay Measures 


2 


1.285 


1.58 




Groups * Pre/Post/Delay 


2 


1.945 


2.39 




Error (SB/A) 


87 


.815 




Exciting 


Groups (Teachers/Resource Professionals) 


1 


8.045 


*17.97 




Pre/PosUDelay Measures 


2 


1.708 


*6.09 




Groups * Pre^ost/Delay 


2 


.746 


2.66 




Error (SB/A) 


87 


.280 





Note. *g<.05. 



Table 5 



Levels of Confidence in Tea ching Environmental Science: Means and Standard Deviation!; for 
Pre/Pnst/Delay . 









Pre 




Post 


Delay 


Variable 


Group 


M 


SD 


M 


SD 


M 


SD 


Using a variety of 


Teacher 


4.15 


0.70 


4.62 


0.55 


4.62 


0.55 


teaching approaches 


Resource 


3.37 


1.12 


3.72 


0.67 


4.05 


0.86 


Teaching process skills 


Teacher 


3.27 


0.79 


4.21 


0.59 


4.35 


0.59 




Resource 


2.89 


1.13 


3.28 


0.58 


3.42 


0.89 


Adapting what you are 


Teacher 


4.00 


0.65 


4.59 


0.56 


4.67 


0.48 


teaching to a specific 
age group 


Resource 


3.53 


1.07 


3.72 


0.75 


3.58 


0.77 


Managing students 


I'eacher 


4.35 


0.77 


4.65 


0.54 




— 




Resource 


3.61 


0.78 


3.78 


0.65 


— 


— 


Managing materials 


Teacher 


3.88 


0.84 


4.35 


0.73 








Resource 


3.78 


0.73 


4.00 


0.59 







Asking effective 


Teacher 


3.62 


0.74 


4.21 


0.69 


4.35 


0.85 


questions 


Resource 


3.39 


0.70 


3.56 


0.62 


3.89 


0.81 


Measuring student 


Teacher 


3.50 


0.90 


4.29 


0.72 


4.32 


0.81 


learning using methods 
other than traditional 
tests or worksheets 


Resource 


2.89 


0.96 


3.61 


0.79 


3.68 


0.75 


Your own level of 


Teacher 


2.85 


0.80 


3.88 


0.64 






knowledge about 
environmental science 


Resource 


3.78 


0.65 


4.22 


0.65 







Teachers (n=35) Resource Professionals (n=19) 
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Table 6 



Qiie.-B et ween-_Qiie _Within ANQ V A bvLEnviiQiimeiital Science V ariahlft anH Pre. -Pnst(#?) 



Vanable 


Source 


df 


MS 


£ 


Using a vanety of 


Groups (Teachers/Resource Professionals) 


1 


19.961 


*37.51 


approaches 


Pre/Post/Delay Measures 


2 


4.227 


*8.30 




Groups * Pre/iPost/Delay 


2 


.169 


0.33 




Error (SB(A) 


61 


.515 




Teaching Process 


Groups (Teachers/Resource Professionals) 


1 


20.90 


*30.68 


Skills 


Pre/PostDelay Measures 


2 


9.283 


*18.91 




Groups * Pre/Post/Delay 


2 


.850 


1.73 




Error (SB(A) 


89 


.491 




Adapting what 


Groups (Teachers/Resource Professionals) 


1 


23.114 


*34.98 


you are teaching 


Pre/Post/Delay Measures 


2 


1.956 


*5.55 


to a specific age 


Groups * Pre/Post/Delay 


2 


1.205 


*3.42 


group 


Error (SB(A) 


89 


.352 




Managing students** 


Groups (Teachers/Resource Professionals) 


1 


*22.548 


*30.18 




Pre/Post^elay Measures 


2 


2.1291 


*9.02 




Groups * Pre/Post/Delay 


2 


.000 


0.00 




Error (SB(A) 


89 


.236 




Managing 


Groups (Teachers/Resource Professionals) 


1 


.579 


0.71 


materials** 


Pre/Post/Delay Measures 


2 


4.804 


*15.90 




Groups * Pre/Post/Delay 


2 


.546 


1.81 




Error (SB(A) 


89 


.302 




Asking effective 


Groups (Teachers/Resource Professionals) 


1 


.329 


*9.93 


questions 


Pre/Post/Delay Measures 


2 


.3.06 


*9.76 




Groups * Pre/TPost/Delay 


/. 


.608 


1.94 




Error (SB(A) 


89 


.314 




Measuring student 


Groups (Teachers/Resource Professionals) 


1 


21.127 


*30.68 


learning using 


Pre/Post/Delay Measures 


2 


3.756 


*18.91 


methods other than 


Groups * Pre/Post/Delay 


2 


1.18 


1.73 


traditional tests or 


Error (SB(A) 


89 


.500 




worksheets 










Your own level of 


Groups (Teachers/Resource Professionals) 


1 


9.9148 


*12.71 


knowledge about 


Pre/Post/Delay Measures 


2 


18.0617 


*73.27 


environmental 


Groups * Pre/Post/Delay 


2 


2.5535 


*10.36 


science** 


Error (SB(A) 


89 


.2465 




Note. .05. 


**Reflects results of data collected for Pre/Post only 
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Table 7 



Correlation Analysis for Level of Confidence in Partnerintr Pre (#] I'l and Post r#8^ 



Group 


M 


Pre 

SD 


M 


Post 

SD 


Spearman 

Coefficient 


Teachers 


1.85 


0.78 


1.11 


0.39 


0.24* 


Resource Professionals 


2.22 


1.04 


1.36 


0.58 


0.44 


Total Group 


1.98 


0.90 


1.20 


0.48 


0.37 



Note: * g < .05. 
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